Harnessing big data
for agricultural excellence

Part 1: Understanding big data in agriculture

noshadow

Marie-Hélène Burle

Simon Fraser University’s Big Data Hub &
BC Centre for Agritech Innovation

November 20, 2024


frontlogofooter

Who we are

Simon Fraser University

SFU hosts the Cedar supercomputer—a cluster of 100,400 CPUs and 1,352 GPUs soon to be replaced by an even larger computer cluster

Simon Fraser University

SFU also works with the Digital Research Alliance of Canada to offer researchers large amounts of computing power to solve challenging data and technology problems, as well as training to optimize their solutions

SFU’s Big Data Hub

Since 2016, Simon Fraser University’s Big Data Hub has been offering workshops, events, and consulting services to researchers and industry partners helping them remain at the top of the fast evolving data landscape

noshadow

BC Centre for Agritech Innovation

Since 2022, SFU BCCAI has been helping small and medium enterprises in the farming industry to embrace technology driven solutions

BC Centre for Agritech Innovation

support Agritech projects training Training & upscaling network Agritech network



Goals for this workshop


Session 1

Today

A (hopefully) friendly lecture to:

  • Demystify big data
  • Demonstrate the critical importance of big data in agriculture and farming



Session 2

Tomorrow at 11am in the Mount Baker Room

An interactive workshop to:

  • Brainstorm on how big data can benefit your operation
  • Help you make the transition to smart farming

noshadow

What is big data?

noshadow

The 3 “V”: Volume

Before

Farmers were taking measurements (e.g. on soil moisture) manually creating low volumes of data







The 3 “V”: Volume

Before

Farmers were taking measurements (e.g. on soil moisture) manually creating low volumes of data

Now

Internet of Things (IoT) (e.g. thousands of soil moisture sensors) collects large volumes of data



The 3 “V”: Variety

Before

There was a limited set of data a producer could collect








The 3 “V”: Variety

Before

There was a limited set of data a producer could collect

Now

There are so many different types of data (e.g. satellite images, market data gathered from internet browsing…)



The 3 “V”: Velocity

Before

A farmer could only gather so much data, even with a lot of employees








The 3 “V”: Velocity

Before

A farmer could only gather so much data, even with a lot of employees

Now

Data is generated in real time and accumulates at high speed


Why has big data become so essential?

All this data is key to the development of artificial intelligence (AI)

so…

What is AI?

AI

Very loosely, you can think of neural networks (the most powerful form of AI) as an attempt to create a computer model that mimics the brain

Biological neurons

Neural network

AI

In traditional computing, a programmer writes code that gives a computer detailed instructions of what to do

These instructions are called a program






Some action

AI

With neural networks, instead of writing a program, a programmer writes a model, then feeds it lots of data and the model changes little by little over time

The model “learns” thanks to this data

Simplilearn has a video explaining how neural networks work in 5 min

AI

This learning is nothing magical: some numbers in the model get tweaked a tiny bit, with each new piece of data, to make the model a little bit better

noshadow

From xkcd.com

AI

To get a very good model at the end—one that can write human language like ChatGPT for instance—you need A LOT OF DATA

AI: an example

You want a program able to detect tomatoes in an image

As human, this is straightforward

Yet, for a traditional approach, this is truly impossible because there are too many factors (location of the tomato in the image, quality of the picture, colour of the tomato…)

AI: an example

By feeding a very large number of images with and without tomatoes to a neural network we can train it to recognize tomatoes in images that it has never seen

Lawal, M.O. Tomato detection based on modified YOLOv3 framework. Sci Rep 11, 1447 (2021). https://doi.org/10.1038/s41598-021-81216-5

Why now?

The idea is not new, but it is only recently that we have had enough computing power, internet connectivity, and storage capacity to implement it

Big data and AI in agriculture

Decision making

Before

Farmers had to make decisions as best they could based on their experience and their limited data







Decision making

Before

Farmers had to take decisions as best they could based on their experience and their limited data

Now

Farmers can use powerful models to make informed decision in real time. This can be followed by the automation of some action (e.g. watering)

Resources

Getting in touch

Understanding neural networks

To go a bit further than the video mentioned earlier, 3Blue1Brown by Grant Sanderson has a series of 4 videos on neural networks which is easy to watch, fun, and does an excellent job at introducing the functioning of a simple neural network

Literature

Open-access preprints:

Arxiv Sanity Preserver by Andrej Karpathy
ML papers in the computer science category on arXiv
ML papers in the stats category on arXiv
Distill ML research online journal

Acknowledgements






Carson Li (BCCAI) suggested an outline for this talk







Ian Chan (BCCAI) provided copious feedback



Feedback

Please give us feedback by scanning the QR code:

noshadow
Thank you!


Diagnosing and implementing big data solutions

Join us tomorrow at 11am in the Mount Baker Room for our 2nd session

We will have an interactive workshop to:

  • Brainstorm on how big data can benefit your operation
  • Help you make the transition to smart farming


If you are unable to attend, you will find the slides here tomorrow, but it will be an interactive clinic with most of the material covered in the activity

Questions?